Cinema and the Code 


hat are the implications of digital imaging 
for the evolution of cinematic language? Since 1986, Peter 
Weibel, Steina and Woody Vasulka and I have been meeting 
to discuss that question [1]. We thought our talks might be- 
come a book, whose subject Weibel conceived as “the evo- 
lution of the image through the digital image”. What follows 
is an outline of our conversations, assembled for this publi- 
cation from 200 pages of transcript. It is in every sense a first 
draft, a working paper. We are quite aware of the problem- 
atic nature of our discourse, especially in the cursory form 
presented here. Every conclusion is vulnerable to criticism, 
which we welcome. We are certain of only one thing: that 
these questions are important and need to be explored. 

The subject of ‘digital imaging’, we agree, exists in the 
context of both video and the computer (different only in 
the source of the image and the possibility of real time 
operation) and covers the generic areas of image pro- 
cessing, image synthesis, and writing or organizing digital 
code in a procedural or linguistic fashion [2]. But in every 
case when we refer to the phenomenology of the moving 
image, we call it cinema. For us it is important to separate 
cinema from its medium, just as we separate music from par- 
ticular instruments. Cinema is the art of organizing a stream 
of audiovisual events in time. It is an event-stream, like music 
(3]. There are at least four media through which we can 
practice cinema—film, video, holog-aphy and structured 
digital code—just as there are many instruments through 
which we can practice music. Of course each medium has 
distinct properties and contributes differently to the theory 
of cinema, each expands our knowledge of what cinema can 
be and do. Each new medium modifies and extends the lin- 
guistic possibilities of the moving image, subsuming the syn- 
taxes of previous media without negating them. 

Thus, the basic phenomenology of the moving image— 
what Vasulka calls “the performance of the image on the sur- 
face of the screen”—remains historically continuous across 
all media. Digital code, for example, has radically altered 
the epistemology and ontology of the moving image but has 
not fundamentally changed its phenomenology. There are 
no digital images that have not been prefigured in painting, 
film and video. With the code we can only summarize them, 
elaborate and unfold them or exercise modalities. Vasulka 
calls the code a variation machine. There are no new classes 
of images, there are only new variations and new epistemo- 
logical and ontological conditions for generating and wit- 
nessing those variations. Each new medium of the future, 
says Vasulka, can only “play host to the phenomenology of 
the moving image”, which will evolve through that medium 
to the next, accumulating the language of each. 
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Weibel puts it this way: a me- 
dium is “a corpus of aesthetic 
strategies” inherited from pre- 
vious media. In the 1920s math- 
ematicians attacked the prob- 
lem of foundations: What was 
pure logic? What was an axiom? 
Today the answers to those 
questions are implemented in 
the computer. Logical con- 
cepts have become instrumen- 
tal, they have become parts of 
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machines. And any machine 
element, says Weibel, is noth- 
ing but a physical implementa- 
tion of a formal device. It im- 
plements mental strategies into 
something physical. (This is 
what Buckminster Fuller meant 
when he defined technology as 
“instrumented or documented 
intellect”). Similarly, aesthetic 
strategies invented 100 years 
ago in photography and 
cinema—scaling, perspective, positive/negative reversals, 
wipes, mattes—have now become machine elements whose 
operations are trivially invoked through the preset button. 
It is a question of primitives. The code is a metamedium: 
through it, high-level aesthetic constructs from previous 
media become the primitives of the new medium. This 
influences which aesthetic strategies will be emphasized. 
When a strategy that was possible but difficult in film be- 
comes a preset button in video or a command in computer 
graphics, it tends to be used more frequently. But that does 
not make it more meaningful. The challenge is to turn ‘ef- 
fects’ into expressions, into syntactical units of meaning. 
This raises the question, How has the corpus of aesthetic 
strategies inherited in a medium like photography or film 
transferred over to electronic media and especially to the 
code? Things are possible in the code that were not possible, 
or at least not easy, in film and video. Only by comparing 
formal devices developed in one medium to other devices 
developed in other media can we arrive at criteria for eval- 
uating artistic achievement. Have the syntactical and linguis- 
tic possibilities of the digital image been identified and 
elaborated in practice? We think not—at least, not very 
often. We rarely find them in the work that is otherwise ad- 
mired in the name of the medium. People praise a particu- 
lar work of ‘video’ or of ‘computer art’, and yet we find in 
this work no definitory elements of video or of the code. It 
may be great cinema but it is not great electronic cinema. We 
are not arguing for exclusivity or essence. We are not trying 
to be the Clement Greenberg of the code. The phenome- 


the formal possibilities of digital 
imaging are employed as syntacti- 
cal or linguistic elements, not 
simply as ‘special effects’. Four 
digital imaging techniques are dis- 
cussed as possibilities for a new 
syntax and, hence, for the expan- 
sion of cinematic language. 
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nology of the moving image remains 
constant across all media, but each 
new medium brings about a shift of 
emphasis or accent. Through the 
code, we can unfold the potential of 
formal strategies that were possible 
but limited in previous media, thereby 
expanding the richness of cinematic 
language. 

Vasulka asks, “Who creates the lan- 
guage ofa medium?” Weibel responds 
by quoting Heidegger: “Man is but a 
guest in the house of language.” Va- 
sulka agrees. All possibilities of a sys- 
tem, he says, are contained within that 
system. We are not free to invent the 
language of film, video or computer. 
The language already exists in the sys- 
tem. Our task is to discover it, identify 
it, draw it out and name it, put a 
nomenclature on it. Vasulka has built 
his machines in order to discover ‘the 
language’ in them, which could be 
found only through dialogue with the 
machines. He points out that this is 
not unique to electronic cinema. Film 
language also arose from a similar sys- 
temic understanding. As a syntactic 
device, the cut, the edit, is machine- 
bound. It is the only way to splice film. 
The most important figures in the his- 
tory of film are those who elaborated 
its syntactic or linguistic potential. 
This is our criterion for artistic 
achievement in the new medium: to 
what extent does the artist articulate 
and develop the formal possibilities of 
the system as syntactical or linguistic 
elements? To what extent does the art- 
ist transform effects into expressions? 

It is a question not only of the evo- 
lution of cinematic language, but of 
human perception itself. Human 
vision, Weibel points out, has always 
been ‘machine-assisted’. The inven- 
tion of perspective, for example, was 
machine-dependent. It was derived 
from optical instruments. Durer’s 
boxes were in this sense ‘machines’. 
They implemented physically what 
then became formal strategies. With 
the help of this machine we could in- 
vent perspective. (Weibel thinks this 
curious. Why did it take so long?) Sim- 
ilarly, Vermeer, under the influence of 
Spinoza and the science of optics in 
the seventeenth century, created 
paintings that were not initially seen as 
poetic. They were regarded more as 
scientific research. (In the nineteenth 
century, Proust, influenced by photog- 
raphy, ‘rediscovered’ Vermeer, now 
regarded as a poet. The computer is to 
the artist of today as the lens was to 
Vermeer.) The Impressionists, too 
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were following theories, not subjective 
experience. Impressionism was based 
on color theory: three different colors 
produce a fourth impression. An opti- 
cal theory of color, says Weibel, is also 
a machine, a mental machine, like a 
Turing machine. Thus we have sub- 
stantial evidence that the evolution of 
vision is dependent on machines, 
either mental or physical. It has come 
to the point that it is no longer pos- 
sible to suppress the machine part of 
it: first there was the camera, now the 
computer. This is significant, Weibel 
thinks, because art always tries to sup- 
press the influence of the machine ele- 
ment in the work itself. It is not art if 
the technology is too apparent. But 
the issue here is not art, it is language 
and perception. They co-evolve only 
to the extent that the syntactic possi- 
bilities of technological systems are 
made the subject of aesthetic inquiry. 

The following formal possibilities of 
digital imaging are available for ar- 
ticulation as syntactic elements or 
linguistic primitives: (1) image trans- 
formation, (2) parallel event-streams, 
(3) temporal perspective and (4) the 
image as object. 


IMAGE 
TRANSFORMATION 


If mechanical cinema is the art of 
transition, electronic cinema is the art 
of transformation. Film grammar is 
based on transitions between fully 
formed photographic objects called 
frames. It is done primarily through 
that collision of frames called the cut, 
but also through wipes and dissolves. 
In electronic cinema the frame is not 
an object but a time segment of a con- 
tinuous signal. This makes possible a 
syntax based on transformation, not 
transition. Analog image processing is 
one vehicle of this particular art—for 
example, scan processors. But it be- 
comes even more significant in digital 
image synthesis, where the image isa 
database. One can begin to imagine a 
movie composed of thousands of 
scenes with no cuts, wipes or dissolves, 
each image metamorphosing into the 
next. 

A cutis a cut, but a transforming or 
metamorphosing operation is open- 
ended. There are infinite possibilities, 
each with unlimited emotional and 
psychological consequences. Meta- 
morphosis is not unique to digital im- 
aging; it is a familiar strategy in hand- 
drawn animation. What is unique is 


the special case of photoreal metamor- 
phosis. It is one thing for a line draw- 
ing or fantasy painting to meta- 
morphose, quite another for a 
photographically ‘real’ object to do so. 
This is theoretically possible in me- 
chanical cinema and has been pre- 
figured (but never fully realized) in 
hand-drawn animation, where it is so 
difficult and time consuming that it is, 
for all practical purposes, impossible. 
It is possible digitally, because the 
code allows us to combine the subjec- 
tivity of painting, the objectivity of 
photography and the gravity-free mo- 
tion of hand-drawn animation. 

Steina points out that there are two 
kinds of transitions based on the cut, 
and these require different kinds of 
metamorphoses. One moves us to a 
different point of view in the same 
space/time, the other moves us to a 
different space and/or time. In flash- 
backs (cinematic memory), either a 
matte is used within the frame or the 
whole frame dissolves. With the code, 
a part of the frame can metamor- 
phose. This implies an expanded cine- 
matic language of simultaneity. 


PARALLEL 
EVENT-STREAMS 


With the arrival of electronic cinema 
it became apparent that film grammar 
was limited in what might be called its 
vocabulary of tenses—for the most 
part it was ‘meanwhile’ or ‘after’. For 
example, simultaneous events are 
traditionally signified through cross- 
cutting, or what is known as parallel 
montage. But, Weibel notes, there was 
never a formal distinction between a 
cut to a different position in space/ 
time (say, between people in conversa- 
tion) and a cut between different 
spaces or time. The distinction has al- 
ways been logical or inferential (as in 
parallel montage), never formal. Digi- 
tal code offers formal solutions to the 
‘tense’ limitations of mechanical cin- 
ema. Past, present and future can be 
spoken in the same frame at once. 
There are at least three possibilities: 
superimpostion (overlay), or simulta- 
neous but spatially separate event- 
streams that are either framed or un- 
framed. Superimposition has been 
explored extensively in experimental 
film, notably by Stan Brakhage. His 
work is the closest cinema has come to 
the Joycean text. In such work it is not 
always possible to identify consciously 
each image-stream, just as it is often 
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impossible to distinguish every voice 
in a musical composition, One is dis- 
turbed by this only if one is unfamiliar 
with it. Once one learns to read it, the 
dense text is a pleasure. Digital code 
offers possibilities of image-overlay 
whose linguistic potential we have not 
begun to explore. 

The second possibility is more fa- 
miliar: framed parallel event-streams, 
such as split screens in film (optical 
printing) or floating imageplanes in 
video, done with digital effects devices 
such as ADO or Quantel. But there is 
also the possibility of unframed parallel 
events occupying different areas of a 
single image. This can best be seen in 
the work of the Vasulkas, for example, 
where pointillist textures move inde- 
pendently in separate areas of the 
frame. Different zones of the image 
are activated in different ways in 
parallel. The Vasulkas accomplish this 
through digital image processing. But 
image synthesis, through a variation 
on metamorphosis, would provide un- 
limited possibilities for unframed but 
separate parallel event-streams in a 
single frame. 

Below, in a discussion of the image 
as object, I shall have more to say about 
parallel event-streams. Meanwhile, 
consider that simultaneity enlarges 
our concept of a cinematic event. Wei- 
bel puts it this way: whereas first we 
had the industry of the moving image, 
today we have the industry of the ac- 
celerated image. If there are three 
image-planes instead of one, the infor- 
mation conveyed within the overall 
frame is tripled, and, furthermore, 
each succeeding image destroys the 
meaning of the previous one. The in- 
formation is accelerated so much in 
perspective and in all other ways that 
the value of ‘the image’ is replaced by 
the value of the image-gestalt or 
image-field. 


TEMPORAL 
PERSPECTIVE 


“The history of every art form”, wrote 
Walter Benjamin, “shows critical 
epochs in which a certain art form 
aspires to effects which could be fully 
obtained only with a changed techni- 
cal standard, that is to say, in a new art 
form” [4]. Weibel pursues this logic in 
reverse, working backward from the 
digital image to find desire for its 
powers in art history. He begins by 
noting that Renaissance perspective 
was always at eye level with one point 


of view and one vanishing point. By 
1850, photographers were climbing 
onto Parisian rooftops and shooting 
down into streets. Twenty years later, 
Odilon Redon painted a_balloon- 
suspended eye moving up into the 
sun. Perspective as no longer bound to 
a static point of view. It had become 
free-floating. In the same period, the 
German Romantic painter Kaspar 
David Friedrich painted mountain 
shadows falling at an angle different 
(that is, displaced in time) from that 
of the impinging sunlight. Other 
examples are found in the work of El 
Lissitsky and the Cubo-Futurist move- 
ment. Painting, influenced by photog- 
raphy and cinema, introduced multi- 
ple points of view and implied time. 

And what did cinema do with per- 
spective? Not much. Bound to psycho- 
logical realism, it exploited it only spa- 
tially, mainly through deep focus 
(Eisenstein, Welles, Renoir), never 
temporally. Only in experimental cin- 
ema was temporal perspective ex- 
plored in any serious way at all—the 
outstanding example being the work 
of Michael Snow, such as La Region 
Centrale and Back and Forth. But with 
the advent of the code, the emphasis 
on perspective returns. Moving-image 
art can now embrace it in an emphatic 
way. When the image is a three-dimen- 
sional database, perspective becomes 
a temporal as well as spatial phenom- 
enon. It is a strategy that is intrinsic to 
the code. Painters, photographers and 
filmmakers could not realize the full 
potential of this desire. But now we 
can unfold and elaborate that which 
could only be indicated in earlier 
media. 

Vasulka notes that, if we remove the 
two cinematic vectors from earth to 
space and establish the principle of a 
point in space, we arrive at two possi- 
bilities: first, cinema looks from one 
point to infinity in a spherical point of 
view. That is one vector, we shall say. 
The other is the opposite: one looks 
from each point in space towards a 
single point. If all these poimts are in 
motion around one point, that is the 
space in which ideal cinema operates. 
But as long as we are talking about psy- 
chological realism we will be bound to 
an eye-level cinema. 


THE IMAGE AS OBJECT 


There are three technologies through 
which the image can become an 
object: image processing, image syn- 


thesis, and three-dimensional dis- 
play—either binocular (stereoptic) or 
holographic. The code is responsible 
for the first two and may be partially 
involved in the third. This is another 
aspect of parallel event-streams. We 
recognize cinema as frame-bound and 
frame-unbound. Mechanical cinema 
is characterized primarliy by its reli- 
ance on the frame. It cannot leave the 
frame unless a special effort is made 
through optical printing. But with 
code it becomes a trivial matter to re- 
move the image from the frame and 
treat it as an object, an image-plane, 
because those tools have no capacity 
to deal with the geometry of the image 
itself: they deal only with its location 
or position (its ‘address’) within the 
larger frame. The use of framed paral- 
lel events points to new narrative pos- 
sibilities, new semiotic strategies—for 
example, the possibility of a previous 
or future event appearing spatially be- 
hind or in front of a current event 
within the same frame. There is always 
a pending image. Editing can be 
avoided entirely—as Vasulka did in his 
1987 work Art of Memory. He points out 
that, through hierarchies of image 
planes in particular arrangements ‘in 
a mental space’, future and past tenses 
may be suggested. As already men- 
tioned in the discussion of parallel 
event-streams, conventional film lan- 
guage is rather inarticulate in this re- 
spect. There is no temporal eloquence 
in film. But digital video suggests the 
possibility of establishing one image- 
plane as ‘present’ with other time- 
frames visible simultaneously within 
the frame. This would extend the pos- 
sibility of transfiguration (metamor- 
phosis) into a narrative space com- 
posed of layers of time, either as 
moving or still images. Ed Emshwil- 
ler’s Sunstone was one of the first works 
to explore these possibilities. In it the 
image becomes object, and it has both 
framed and unframed parallel event- 
streams. 

When image becomes object in a 
stream of parallel events, the realm of 
psychological realism or  photo- 
graphic truth is abandoned. The 
frame-bound photographic image 
brings us truth. But three image- 
planes within a frame lose what Va- 
sulka calls “the aura of truth”. We 
detach ourselves from them psycho- 
logically. Will it be possible to con- 
struct a psychological space in a 
language of frame-unbound parallel 
event-streams? 

For Weibel, all this raises a 
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fundamental challenge to the meto- 
nymic nature of cinematic language. 
He invokes the name of Roman Jakob- 
son, who argues that there are only two 
fundamental operations in language: 
metaphor and metonymy. And the 
language of cinema is not metaphoric, 
it is metonymic. It is the language of 
the part for the whole. Al] cinematic 
images are contingent. The frame, 
said Jakobson, is always part of an un- 
seen whole. At its fundamental syn- 
tactic level—the level of cutting, of 
editing, of bringing spaces together— 
the filmic language game is meto- 
nymic. In the service of psychological 
realism, conventional editing recon- 
structs ‘real’ time and ‘real’ space, fol- 
lowing logical causal chains by me- 
tonymic association. Experiments like 
Last Year at Marienbad were attempts to 
transcend that limitation within psy- 
chological narrative. But in the elec- 
tronic image there is no need to make 
a Marienbad, because it is clear that we 
no longer have that constancy of time 
and space. Once an image-object is set 
against a reference, the metonymic 
tension is lost. Objectifying the image 
within the frame puts it in a different 
time zone. Metonymy becomes prob- 
lematic. On the one hand, such con- 
structs are not metonymic because the 
space they occupy is not ‘natural’. The 
image-object is not part of the whole; 
itis no longer contingent. But it is not 
metaphoric either. It is something 
new. We do not know what it is. It 
might still function metonymically, 
but in a different way. This isan impor- 
tant area that is wide open for aes- 
thetic exploration. 

The second level of the image as ob- 
ject is achieved through digital image 
synthesis. Here, because it is a three- 
dimensional database, we can control 
not only the location of the image- 
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object within the frame but also its 
perspective, its angle of view, its 
geometry. As a result, the synthesized 
image becomes truly an object, the wit- 
ness becomes a ‘user’, and the relation 
between them becomes not obser- 
vation but interaction. Jean-Louis 
Baudry argues that, in the cinema of 
psychological realism, the primary 
identification of the spectator is not 
with the characters but with the cam- 
era itself [5]. But in interactive image 
synthesis, the spectator 7s the camera. 
Since it is not separate from the scene 
it surveys, the virtual camera is neither 
a voyeur nor an instrument of surveil- 
lance. “It is a point of view that is active 
within the scene”, writes Catherine 
Richards. “Not only can this camera 
(the user) direct its own looking, it can 
be sensed, responded to, and rep- 
resented in the scene: it sees and is 
seen” [6]. 

The third level of the objectifica- 
tion of the image is realized through 
three-dimensional display. Whether 
through holography or binocular 
(stereoptic) technology, cinema is 
moving from the two-dimensional 
image on a screen to the three-dimen- 
sional object in space. Today cinema 
represents reality; tomorrow it will be 
reality. Already with stereoptic tech- 
nology the image becomes an object. 
And in Scott Fisher’s virtual envi- 
ronment project of the U.S. National 
Aeronautics and Space Admini- 
stration (NASA) (combining a three- 
dimensional database with stereo 
vision in a wraparound head-mounted 
display), cinematic space becomes a 
place to live. An unframed image is 
not an image, Vasulka points out, it is 
an object in space: “It forces you to 
deal with air.” It is no longer a repre- 
sentation but the thing itself. Vasulka 
notes that different understandings of 


reality and truth are implied by the 
representational image and by an 
object in space, no matter how insub- 
stantial that object may be. Three- 
space cinema, he suggests, is more like 
theatre. In two-space cinema there it 
truth but no reality. In theatre there is 
reality but no truth. 
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